Papers with sampling process
CDAˆ2: Counterfactual Diffusion Augmentation for Cross-Domain Adaptation in Low-Resource Sentiment Analysis (2025.coling-main)
Copied to clipboard
| Challenge: | Domain adaptation is widely employed in cross-domain sentiment analysis, but concerns have been raised regarding their robustness and sensitivity to data distribution shift. |
| Approach: | They propose a framework CDA2 for cross-domain adaptation in low-resource sentiment analysis which employs counterfactual diffusion augmentation. |
| Outcome: | The proposed framework generates high-quality counterfactual target samples and achieves state-of-the-art performance on benchmark datasets. |
Uncertainty Modeling for Machine Comprehension Systems using Efficient Bayesian Neural Networks (2020.coling-industry)
Copied to clipboard
| Challenge: | Neural approaches have improved machine comprehension tasks, but models often operate as a black-box, resulting in lower interpretability. |
| Approach: | They propose a hybrid approach to quantify model uncertainty using Bayesian weight approximation and boost up inference speed by 80% relative to test time. |
| Outcome: | The proposed approach boosts inference speed by 80% relative to the previous approach and is applied to a clinical dialogue comprehension task. |
Deep Reinforcement Learning with Hierarchical Action Exploration for Dialogue Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to improve dialogues with random sampling are inefficient due to the large number of eligible responses with high action values. |
| Approach: | They propose a dual-granularity Q-function that extracts actions based on a grained hierarchy . they use offline RL and learn from multiple reward functions designed to capture emotional nuances in human interactions. |
| Outcome: | The proposed approach outperforms baselines across automatic metrics and human evaluations. |
CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown significant potential in code generation, but they also present challenges regarding the protection of Intellectual Property (IP) related to model architectures, weights, and training data. |
| Approach: | They propose a multi-bit watermarking technique that embeds additional information to preserve provenance details, such as the vendor ID of an LLM. |
| Outcome: | The proposed technique preserves provenance details while maintaining syntactical correctness of generated code. |
Few-shot Temporal Pruning Accelerates Diffusion Models for Text Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing acceleration methods for text generation ignore the importance of the distribution of sampling steps, resulting in slow sampling rates. |
| Approach: | They propose a technique to accelerate diffusion models for text generation without additional training by using a Bayesian optimization approach. |
| Outcome: | The proposed technique achieves 400x acceleration even with minimal sampling steps after down to less than 1 minute of optimization yielding a competitive performance even with minimum sampling steps. |
DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to text generation use discrete text within a continuous diffusion space, which incurs substantial computational overhead during training and results in slower sampling speeds. |
| Approach: | They propose a soft absorbing state that facilitates diffusion models in learning to reconstruct discrete mutations based on the underlying Gaussian space. |
| Outcome: | The proposed method accelerates training convergence by 4x and generates samples of similar quality 800x faster, rendering it closer to practical application. |
Improving Consistency for Text Summarization with Energy Functions (2023.findings-emnlp)
Copied to clipboard
Qi Zeng, Qingyu Yin, Zheng Li, Yifan Gao, Sreyashi Nag, Zhengyang Wang, Bing Yin, Heng Ji, Chao Zhang
| Challenge: | Current abstractive summarization models generate inconsistent content due to the inherently noisy dataset and the discrepancy between maximum likelihood estimation based training objectives and consistency measurements. |
| Approach: | They propose a new consistency taxonomy that categorizes inconsistent content into faithfulness, factuality, and self-supportiveness. |
| Outcome: | Experiments on XSUM and CNN/DM datasets show that EnergySum mitigates the trade-off between accuracy and consistency. |
Self-Para-Consistency: Improving Reasoning Tasks at Low Cost for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that self-consistency decoding can improve performance for complex reasoning tasks with large language models. |
| Approach: | They propose a self-consistency decoding strategy that generates multiple paraphrases for each test question and then generates reasoning paths for the original and all the paraphrased questions based on greedy decoding. |
| Outcome: | The proposed strategy reduces the sampling number and improves performance on complex reasoning tasks. |
Towards Informative Few-Shot Prompt with Maximum Information Gain for In-Context Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language models (LLMs) have the capability to engage In-context Learning (ICL) however, this particular learning paradigm suffers from high instability stemming from factors such as input distribution, order and prompt formats. |
| Approach: | They propose to quantify the information gain obtained in prediction after observing a given example candidate and to sample those with maximum IG. |
| Outcome: | The proposed method can yield an average relative improvement of 14.3% across six classification tasks using three LLMs. |
DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are susceptible to generating harmful content when prompted with carefully crafted inputs, a vulnerability known as LLM jailbreaking. |
| Approach: | They propose an end-to-end generative approach for jailbreak rewriting inspired by diffusion models that uses a sequence-tosequence (seq2sequ) diffusion model as a generator, conditioning on the original prompt and guiding the denoising process with a novel attack loss. |
| Outcome: | Experiments on Advbench and Harmbench show that the proposed method outperforms autoregressive jailbreak models across evaluation metrics including ASR, fluency, diversity and diversity. |
A Theory of Response Sampling in LLMs: Part Descriptive and Part Prescriptive (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly utilized in autonomous decision-making, where they sample options from vast action spaces. |
| Approach: | They propose to use heuristics to sample LLMs to identify a prescriptive component and a descriptive component to represent a concept. |
| Outcome: | The proposed model is compared with human decision-making models in public health, economic trends and other real-world domains to show that it is biased. |
Efficient Safety Alignment of Large Language Models via Preference Re-ranking and Representation-based Reward Modeling (2025.acl-long)
Copied to clipboard
| Challenge: | Existing safety alignment methods for Large Language Models (LLMs) face the distribution shift issue, which requires significant computational resources. |
| Approach: | They propose a framework that leverages the model’s intrinsic safety judgment capability to extract reward signals, which are then used to calculate label confidence for preference reordering. |
| Outcome: | The proposed framework improves safety performance while avoiding 300x computational overheads. |